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(54) Adaptive routing system and method for Qos packet networks 



(57) A packet network employs routers that deter- 
mine network routing based on quality of service (QoS) 
provisioning parameters and network topology informa- 
tion. QoS provisioning parameters are provided to each 
router from a network management database, and the 
network topology information is determined from a link 
state database of the router. The link state database 
may include network topology information collected by 
the router in accordance with the open shortest path 
protocol (OSPF). A network link, router, or other node 
failure initiates a new path-selection process. First, a 
temporary set of provisioning entries may be deter- 
mined with a shortest path first (SPF) routing method. 
Then, the network packet flows may be classified into 
packet flows, real-time and non-real-time, and then as 



packet flows that require reserved bandwidth or that 
may be multiplexed. A multicommodity flow (MCF) rout- 
ing method is then employed to determine an optimized 
set of candidate provisioning entries for the packet flows 
that may be multiplexed. The MCF routing method 
determines new routing for the packet flows based on 
QoS provisioning commitments as parameters. The 
MCF routing method determines the new routing based 
on an optimization criterion, such as maximized reve- 
nue. Once the new routing is determined, routing of net- 
work traffic is enabled by converting the provisioning 
entries into filter rules, which are then loaded into the 
packet classifier of the router. 
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Description 

Field Of The Invention 

[0001] The present invention relates to packet networks, and, more particularly, to adaptive routing of packets in a 
communications network based on quality of service provisioning. 

Description Of The Related Art 

[0002] Packet network service providers, such as those providing Internet Protocol (IP) networks, are presently 
developing and providing differentiated services. In particular, service providers are interested in differentiated services 
as part of provisioning and maintaining networks for customers with a variety of voice and data needs. For example 
virtual private networks (VPNs) allow service providers to establish and tear-down connections for a large number of 
users, while billing only one customer or client (e.g.. a large corporation or government agency) for the service How- 
ever, the customer or client may require a certain amount of guaranteed bandwidth, minimum transport delay, or other 
metric for some or all of the data transported through the VPN. Service providers generally desire a higher revenue for 
data transported through the VPN when a customer demands such differentiated services. 

[0003] In the art, the term "physical path" denotes the path between source and destination pairs for a given com- 
munication stream of data, or packet flow, together with intermediate nodes, if any, and the physical links that connect 
the nodes. In networks of practical size and complexity, many sets of multiple physical paths may exist, each provi- 
sioned to carry one or more corresponding packet flows. The network operator may specify a predetermined bandwidth 
for the corresponding stream or packet flow along each link of the possible physical paths. The specified bandwidth may 
equal the full bandwidth, or some quantity less than the full bandwidth, or no bandwidth at all. Although a physical path 
may exist between a source and destination pair, the full bandwidth of the physical path links may be unavailable for the 
corresponding packet flow. 

[0004] In addition, the term "logical path" (also called a "virtual path") denotes a path between a given source and 
destination pair as defined by connection metrics, such as delay or bandwidth. Logical (virtual) paths are not deter- 
mined purely by physical considerations, but instead are partly defined by parameters that may be specified and/or 
changed. Individual virtual paths may require less than the fully assigned bandwidth physically available on the physical 
links that the paths occupy. Thus, several virtual paths may co-exist along part or all of a physical path, allowing each 
physical link of the network to simultaneously support several virtual paths. 

[0005] Packet networks that provide differentiated services generally include a routing method that allows the net- 
work to determine a logical path through the network from a given source to a given destination. The routing method 
may be part of a process performed in each router of the network. Each router may learn the network topology by 
exchanging link and router information between directly connected routers. Once the router determines the network 
topology, the router may employ a path-selection process that determines the logical path through a network between 
the source and destination pair over the physical links according to predefined connection metrics. Each router deter- 
mines the same logical path through the packet network given the same information. Packets received by each router 
are then routed along the determined path. 

[0006] A standard protocol employed by IP networks to determine network topology and link routing is the Open 
Shortest Path First (OSPF) protocol, as outlined in J. Moy. "OSPF Version 2," Internet Draft, Request for Comment 
(RFC) 2178, July 1997. The OSPF protocol employs the destination (i.e., shortest path to destination) as the metric for 
the path-selection process. However, with destination as the only metric, only a "best effort" is employed to route packet 
flows. Best effort implies that, while the packets are re-routed, no other QoS provisioning commitment, such as mini- 
mum bandwidth, delay or loss, applies to the packet flow. Routers of the prior art may route packets based on other fac- 
tors, such as available bandwidth. However, when the OSPF protocol is employed as the routing method, distribution of 
packet flows over the links is based on shortest path and not provisioned QoS commitments. 
[0007] Routing methods may determine logical paths and allocate corresponding bandwidth to the links in the net- 
work, with such allocation according to the source and destination pair, subnetwork, and class of service This path- 
selection method or determination may be referred to as logical network design. One important element of logical net- 
work design is selection of a set of physical paths through the network having sufficient capacity for carrying the esti- 
mated traffic of a communication stream or packet flow The routing method may base the determination on such factors 
as the network topology, the currently available buffer space at the nodes, and the currently available link capacities 
[0008] Significantly, the network operator may have QoS commitments with its customers, such as guaranteed 
bandwidth or maximum cell-loss probability. The path-selection process may account for such commitments but the 
routing method may require simplifications to characterize variable traffic characteristics in a form that the routing 
method can use for logical path determination. For example, certain networks, particularly high-speed networks may 
support the networking protocol known as asynchronous transfer mode (ATM). Such high-speed networks typically 
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carry multiple services having different traffic characteristics, including both constant bit rate traffic and variable bit rate 
traffic. An important simplification for routing methods considering variable bit rate traffic in networks, such as ATM net- 
works, is provided by the concept of "effective bandwidth." 

[0009] Effective bandwidth of a packet flow defines the bandwidth of the packet flow in terms of the node buffer size 
5 and the average and peak rates of the packet flow, assuming the node operates with a leaky bucket regulator. An effec- 
tive bandwidth may be determined for each packet flow, of a network where the packet flows are multiplexable, the total 
effective bandwidth of all packet flows is less than or equal to the total network capacity, and the packet flows exhibit 
loss-less performance, (i.e., the packet flows may be rearranged over multiple physical paths while maintaining desired 
delay or bandwidth for each packet flow). For networks having both multiplexable and non-muliplexable packet flows, 
w the two types of traffic may be separately analyzed, with portions of the total network capacity allocated to each traffic 
type. 

[0010] Effective bandwidth, and allocation of bandwidth and buffer capacity based on effective bandwidth, is 
described in detail in A. Elwalid et al., "A New Approach For Allocating Buffers and Bandwidth to Heterogeneous, Reg- 
ulated Traffic in an ATM Node," IEEE Journal on Selected Areas in Co mmunications. Vol. 13, No. 6, (August 1995) 
15 1 1 15-1 127 (incorporated herein by reference and referred to as "Elwalid et al."). A routing algorithm that accounts for 
variable bit rate traffic in ATM networks with effective bandwidth, is described in detail in U.S. Patent No. 5,838,663, 
issued on November 11, 1998 entitled "A Method for Admission Control and Routing by Allocating Network Resources 
in Network Nodes", which is incorporated herein by reference. 

[0011] Once logical paths are specified, the routers of the network generally must support at least two types of 
20 messages for the packet flows and network administration: data messages and control messages. Data messages are 
packets containing user information transported between the source and destination pair. Control messages are mes- 
sages interpreted by network elements, such as the routers, to administer connections and exchange network informa- 
tion. These control messages may be employed to ensure that the packets are interpreted and routed in a manner that 
provides the desired commitment of the path-selection process. Control messages may be included in packet headers, 
25 or may be separate and distinct packets. 

[001 2] One approach of the prior art for providing differentiated services employs type-of-service (TOS) bits defined 
in the packet header, or TOS addressed metrics in the packet payload. The TOS bits are interpreted by routers within 
the network in an agreed upon manner so that each network node provides the desired QoS commitments of the serv- 
ice when routing the packet flow. This approach may allow for distribution of each and all resource requests to all nodes 
30 of the network. Alternate approaches may be reservation-based and may employ signaling to reserve network 
resources for the duration of a packet flow, or packet flow aggregates. A protocol to signal reservation of network 
resources in a network is the Reservation Setup Protocol (RSVP). RSVP, as an example, may be used in conjunction 
with service models, such as guaranteed rate and controlled load service models, to request the desired QoS commit- 
ments for certain flows. 

35 [0013] Once the connection is established, the router includes a routing fabric to physically route packets received 
at input ports to specific output ports for transport to the next node or destination. Packet filters may be employed with 
the routing fabric to determine actions to be taken for packets having certain source and destination addresses arriving 
at the router. Actions may be that routing of packets sent from one or more of specified sources is denied (e.g., actions 
for "firewall" or security purposes), or that a specific action is to be taken for that packet having a specified source 

40 address (e.g., actions for routing to a specific, alternate destination). The router may be provided with a table or list of 
filter rules specifying the desired actions for corresponding field addresses. The packet filter allows the router to asso- 
ciate fields of received packets with the table or list entries. 

[0014] Packets comprise, for example, a header and a payload that may include user data. A header of the packet 
includes fields or parameters, such as a source address where the data originates and at least one destination address 
45 where the data is to be routed. Mother parameter in the header may be a protocol type identifying a particular protocol 
employed in the communication network. Source and destination addresses may be logical addresses of end hosts. 
The header may also include, for example, certain types of flags in accordance with protocol type, such as TCP, 
depending upon the receiver or transmitter application. 

[0015] The IP packet header fields typically contain up to 128 bits of parameter information, including source and 
so destination addresses, physical source and destination port numbers, interface number, protocol type, etc. Filter rules 
may also specify, for example, that for received packets with fields in the header specifying a particular destination 
address, the packet should or should not be forwarded through specific output links or output ports. Thus, a variety of 
filter rules may be implemented based on packet field information. For example, such filter rules might be based on 1) 
source addresses; 2) destination addresses; 3) source ports; 4) destination ports; and/or 5) any combination of these 
55 fields. Consequently, such packet filtering may be employed by layer four switching applications. 

[0016] Each of the fields or parameters in the header may be represented as points along an axis of a dimension. 
The general packet classification problem of a packet filter may then be modeled as a point-location in a multi-dimen- 
sional space. One or more field values of the packet define a point in the multi-dimensional space. A packet filter rule 
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associated with a range of values in each dimension defines an object in the multidimensional space. 
[001 7] A point-location algorithm in a multi-dimensional space with multi-dimensional objects finds the object that 
a particular point belongs to. Given a received point EP = E^ . . Erf in a space having D dimensions, the point- 
location algorithm finds one or more of a set of n D-dimensional objects including the point EP (n being an integer 
greater then 0). The general case of D > 3 dimensions may be considered for the problem of packet classification For 
toe special case of two dimensions, the filter rules defined for field ranges are modeled as objects in two dimensions 
forming rectangles in the 2-dimensional space. The specific case of filter rules defined for field ranges that are modeled 
as objects in two d.mensions may correspond to switching of packets through a routing fabric based on source 
address/port and destination address/port. 

!! 018 l ° n °f the , PhySiC3 ' 8nd l09iCal ^ through the network are determined, network resources are reserved 
torough the nodes of the network when a link is established and packet flows between source and destination pairs 
begin. However, when a link is broken, such as by a disconnected T1 , T3, or fiber-optic connection to an interface card 
of toe network node, or when a node fails, such as when a router fails, the routing mechanism must determine alternate 
paths through the network to re-establish the packet flows. 

[0019] Since packet network links are generally in a state of flux, service providers may not be able to effectively 
provide QoS provisioning, and hence receive revenue, when re-routing does not effectively preserve the QoS commit- 
ments to the VPN. Routing mechanisms of the prior art currently route packets through the network based on destina- 
tion address Reservation-based protocols alone may only request bandwidth for a link to be established with no 
guarantee of success of the request. 

Summary Of The Invention 

[0020] The present invention relates to adaptive routing, and more particularly to generating new routing paths 
through a router of a packet network. Generating new routing paths may occur when new paths for packet flows are 
desired because a trigger event, such as a node/link failure or a network topology/provisioning change, occurred in the 
packet network. In accordance with the present invention. 1) network topology information and 2) quality of service 
(QoS) provisioning information are collected for each packet flow through one or more routers of the packet network A 
pato for each packet flow is determined using a general routing optimization method, such as a multicommodity flow 
routing metood based on the QoS provisioning and network topology information. A set of one or more filter rules is 
generated for the router based on the one or more paths for the one or more packet flows passing through the router, 
each fitter rule defining a physical path tor one or more packet flows through the router. A selected filter rule is applied 
to each packet of a packet flow to cause each packet to traverse the corresponding path through the router 
[0021] The present invention may allow for adaptive routing using packet classification with filter rules. New path 
routing may be generated as provisioning entries generated by a multicommodity flow-based routing method using net- 
work topology information and quality of service (QoS) provisioning information. The provisioning entries may then be 
translated into filter rules of a packet classifier that implements the new routing paths, thereby implementing adaptive 
routing in accordance with the present invention. 

Brief Desc ription Of The Drawing s 

[0022] Other aspects, features, and advantages of the present invention will become more fully apparent from the 
following detailed description, the appended claims, and the accompanying drawings in which: 

FIG. 1 shows block diagram of a packet network employing adaptive routing in accordance with the present inven- 
tion; 

FIG. 2 shows a block diagram of an exemplary processing section of a router implementing adaptive routing in 
accordance with the present invention; 

FIG. 3 is an exemplary flowchart for a router implementing adaptive routing in accordance with the present inven- 
tion when a trigger event is detected; 

FIG. 4 shows an exemplary implementation of a packet filter of a packet classifier as shown in FIG. 2; 

FIG. 5 shows a block diagram of an exemplary distributed processing and database (DPD) system as may be 
employed with the present invention; 

FIG. 6 shows an exemplary line format of an OSPF packet showing the position of the type of service bits; 
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FIG. 7 shows an exemplary line format of an advertisement packet based on an OSPF packet that may be 
employed with the present invention; 

FIG. 8 shows a flowchart of a path-selection process that may be employed by the QoS routing module of FIG. 4; 

5 

FIG. 9 shows an illustration of packet filtering for two fields of a packet header corresponding to a point location in 
2-dimensions; 

FIG. 10 shows an exemplary flowchart for the general routing optimization as may be employed by the present 
w invention; and 

FIG. 1 1 shows a flowchart for the step of determining route loss probabilities and network sensitivities of the 
method shown in FIG. 10 with stochastic variables. 

15 Detailed Description 

[0023] FIG. 1 shows block diagram of a packet network 100 employing adaptive routing in accordance with the 
present invention. The packet network 100 includes network routers 102-107, and a network management station 108. 
For convenience of the following described embodiments, the routers are defined as a source router 102 in communi- 
20 cation over a link with destination routers 106 and 107 through intermediate routers 103, 104 and 105. The data path 
established through links between the source router 102 and destination routers 106 and 107 may be employed to 
transfer information, or communication streams of data, by packet sequences defined as packet flows. Each packet of 
the packet sequences may have a header including a source address, source port range, destination address, and des- 
tination port range. 

25 [0024] Each of the routers 102-107 may typically support an Internet Protocol (IP), for example, and provide for 
Quality of Service (QoS) provisioning for links of the network. These routers are interconnected by links, which are 
physical transport media such as T1 s, T3s, or optical fiber (e.g., SONET links such as OC1 , OC3). However, the present 
invention is not limited to these fixed or wired types of networks, and is not limited by link-layer characteristics. Each link 
has a capacity defined in terms of, for example, total available channels, bandwidth, bit-error rate and delay. Each link 

30 capacity provides a portion of total available capacity of packet network 100. 

[0025] Network management station (NMS) 108 (e.g., a central provisioning station for a virtual private network 
(VPN)) allows network administrators to remotely allocate transmission capacity of routers and links between a source 
and a destination pair (source-destination pair) within the network. A given VPN may comprise many source-destination 
pairs. For such purposes, the NMS 108 maintains a network management database 109 of addresses corresponding 

35 to the source-destination pairs, and a pair is also identified by the address fields in the header of a network packet. Cen- 
tral provisioning of packet network 100 also includes QoS provisioning, and so each source-destination pair is assigned 
specified QoS commitments for packet flows corresponding to the source-destination pair. 

[0026] The source-destination pair defines point-to-point traffic through the packet network 100, where "point" 
refers to the ingress (source) or egress (destination) network element. For networks following the OSPF protocol, the 

40 point may typically be a border router for transit traffic. Source router 102 is shown as an ingress point from external 
network N 1t while the destination routers 106 and 107 are shown as egress points to networks N 2 and N 3 , respectively. 
[0027] For packet network 100, packet flows may be classified by header information, and grouped into classes, 
super-classes, or groups of super-classes according to source address ranges and destination address ranges. 
Classes may be associated with packet flows with common QoS commitments. Super-classes, or groups of super- 

45 classes may be associated with virtual private networks (VPNs). Desirably, ranges of source and destination addresses 
are contiguous for each class, super-class, or groups of super-classes. The packet network 1 00 may employ a message 
of a protocol to request resources. Such message may be, for example, Type of Service (TOS) bits of the OSPF proto- 
col in the packet header, TOS addressed metrics of the packet payload, or the control messages of the RSVP protocol. 
Such resource requests may allocate bandwidth for packet flows from the source 1 02 to one or more of the destinations 

so 106 and 107 through the network routers 103, 104, and 105. 

[0028] QoS provisioning specifies certain minimum allowed characteristics of the transmission capacity of the net- 
work to be assigned to provisioned packet flows between source-destination pairs. Transmission capacity may be 
directly related to the capacity of the links in the network, and such characteristics for the packet flows may be band- 
width, delay, and loss (i.e., probability of a lost packet). Such characteristics may be related through available buffers of 

55 the router. The QoS provisioning associated with addresses of each source-destination pair is also maintained in the 
network management database 109. QoS provisioning may also include associated revenue and cost corresponding to 
each QoS provisioning characteristic, which revenue and cost may be related to revenue paid to service providers by 
clients based on minimum levels of QoS characteristics provided to packet flows and the cost of providing network con- 
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nectivity, respectively. 

[0029] Table 1 shows an exemplary provisioning table for QoS 
router in accordance with the present invention. 



link provisioning of a packet flow as employed by a 



Table I 





Router 


Field 


Description 


10 


Router 




Defined traffic class 




Router 


Revenue 


iw iwau>c > ajuc or routing tne packet flow according to a predefined 
metric 


15 


Router 


" VPN-ID 


An identifier for the client or VPN that the packet flow belongs to 






OSPF-ID 


The identifier (ID) of the source router where the packet flow ongtnates 
(here identified as an OS PF protocol router) 


20 




Ingress Interface 


The interface, or port number, of the interface of the ingress router that 
receives the packet flow 


25 


Source 


IP_addr_range 


The range of source addresses that constitute the source of the packet 
flow K 


30 r 








Destination 


OSPF ID 


Tne IU or the destination router that is the destination of the plckcT 


35 


Destination 


Egress interface 


The interface, or port, numbers), of the interface of the egress router 
that provides the packet flow 


j 


Destination 


Next hop 


The address for the node or router that receives the packet flow from the 
egress router 


40 


Destination 


(P_addr-range 


The range of destmauon addresses that constitute the destination of the 
packet flow 



45 Eta add^rZ^iS 1 " ^ PaCke L fl ° W ^ be identified With Class field ' Revenu « values, and VPN-ID 
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Table 2 



Fiplri 


rui iouui i 


1 alt? ^/ f 


Avpranp rsfp nf the* nackot flrt\A/ in Kv/toc nor cornnH nr mau Ko a tnVon rofrocn rata finr tHo loalrvs Ki inlrot 
nvciayc laie ui me (jamvei nuw in uyic?o ptsi acwjiiu, ut may uc a iu rv.cn icircoii lait? iui lite icaivy uuutvel 

regulator 




MaYimi im ni imhor nf nsiflfotc tHat mau Ho cont hofinro o rofrocK r\f tKo Kiir»bai tKo Kt i r*bai rani ilotrtr r\r 

iviaAiinuin nuinucr 01 pad\cis inoi may oe syiu Dciur e a reiresn 01 ine uuCKei ot ine duckci r eguiaior or 
depth 


peak (p) 


Maximum rate of the packet flow In bytes per second. 


min 


Minimum packet size, which may be employed to calculate a maximum or worst case expansion of traffic 
for a packet flow. 


max 


Maximum packet size, which may be employed to determine the number of bytes in the bucket. 


delay 


Maximum sustainable end-to-end delay in microseconds, which may be related to the maximum per hop 
delay, dp (if total maximum number of hops n max is known. 


lossreq 


The maximum sustainable packet loss rate for which the packet flow may operate reliably. 



20 

[0031] The values for the fields shown in Table 2, such as rate, bucket, delay and loss of a leaky bucket regulator, 
may be related to the bandwidth available on specified links. 

[0032] Each of the routers 102-107 comprises a processing section implementing the adaptive routing in accord- 
ance with the present invention. Other routers, not shown, that do not employ the adaptive routing in accordance with 

25 the present invention may exist in the network 100. Each of the routers 1 02-1 07 is connected to at least one other router 
in the network 100. The packet network 100 employs a standard protocol to determine and disseminate information 
regarding the connectivity of the links and routers within the network 100, including information about link capacity and 
available bandwidth, tor example. For the preferred embodiment of the present invention, this protocol for dispersion of 
link and router connectivity information (herein referred to "network topology information") may be provided from the 

30 corresponding function of the Open Shortest Path First (OSPF) protocol. 

[0033] For the packet network 100, each router maintains a database of network topology information. Conse- 
quently, each router knows 1) the existence of each other router in the network; 2) the connections between each router 
in the network, and 3) the identities of routers that interface to other packet networks "outside" of the packet network 
100. In the present context, "outside" may refer to packet networks N 1f N 2 , and N 3 not supporting adaptive routing 

35 and/or part of a different service provider's network. 

[0034] The Open Shortest Path First (OSPF) protocol commonly employed in IP networks includes a Shortest Path 
First (SPF) routing method for the path-selection process. For the preferred embodiments of the present invention, rout- 
ers of network 100 employing adaptive routing in accordance with the present invention determine logical paths based 
on a multicommodity flow (MCF) routing method. The MCF routing method may employ information generated by SPF 

40 routing method of the OSPF protocol as initial provisioning values, and for provisioning values for non-real-time packet 
flows and for real-time, non-multiplexable packet flows. In addition, the routers connected in network 100 that do not 
employ adaptive routing in accordance with the present invention preferably employ a path-selection process based on 
the SPF routing method. 

[0035] The MCF routing method accounts for the QoS provisioning assigned to source-destination pairs when rout- 
45 ing packet flows. Packet flows may be comprised of several different sets of traffic for different users within a VPN. 
Before a link failure, a portion of the link's capacity is allocated to the source-destination pair. However, when the link 
fails, the different sets of traffic desirably have capacity of other links re-allocated to the sets of traffic. In addition, two 
possible classes of routing may be supported by adaptive routing: multi-path routing and single-path routing. Multi-path 
routing occurs when flows of a source-destination pair are split, with different ones of the sets of traffic allocated to dif- 
so ferent new paths, and hence links, through the network. Single path routing occurs when the entire group of sets of traf- 
fic are re-routed over the same new path. 

[0036] If multipath routing exists in the packet network 100, the multi-path routing may be decomposed into single 
path routing (i.e., multi-path from a source to N destinations, N an integer greater than 1 , may be thought of as routing 
over N single paths. In addition, mathematical tools, such as Steiner path trees, heuristic bin packing, randomized rout- 
55 ing, or shortest path spanning trees, may be employed by the algorithms described herein to reduce computational 
complexity resulting from decomposition of multi-path to single path routing. Consequently, the exemplary routing meth- 
ods are described herein for single path routing. The present invention is not so limited, however, and one skilled in the 
art may extend the exemplary routing methods to multipath routing. 
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[0041] 

25 



SS. addrt !° n ' wh . ,le * e P refeTCd embodiments of the present invention are described employing the OSPF 
P ! oto( f mat Ermine and disperse network topology information may Cons* 
quently, one stalled ,n the art may readily adapt the system and methods described herein to other suS nrntn^t 
and/or routing mechanisms. Similarly, white preferred embodiments are described thSemZ Se fpp ro2o 
. asa path-se,ec«on process, other common routing methods may be employed and t SnT^ffot^ 

»o [0039] However, under some network conditions, "loop packets" mav exist fi/ na^tc th=« ~ ♦ 

k • T ° reduce J he number of |0 °P P a <**te that may be formed when the path-selection process is initiated sev 

Kate nT'rnf °^ 3 ^ dia9ram ° f 3n eXemp ' ary pr0CeSsin 9 section of a ">uter, e.g., router 105 employing 
2!^ In nr 9 accoraance witn »» ***** ^ntion. Processing section 200 includes a contro pnSsso? 

£' S!SSSSS^STT 203 / : nput 'r k interface 2M ' buffer secfon 205 - route al,o -£ p = 

te emZSfn l ? » " lm P' ementat,on of *"e processing section and link interlaces of such router mav 
^embodied ,n, for example, a Packet Star 6400 router, available from Lucent Technologies, Inc. of 2 5£j Z 

jmnea. M « ta^ w op**, p*** w, ,» pate Z b£2Z 

empioyed to control resource allocation among VPNs, among services w«hin VPNs'and mn^SSS^ 
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pairs within the service classes. Weights may be calculated by the routing processor 202 and downloaded into the con- 
trol processor 201 and/or scheduler. 

[0045] Control processor 201 detects a condition requiring a new adaptive routing calculation in accordance with 
the present invention, and hence initiates the path selection process to determine the logical paths of source-destina- 

5 tion pairs of the network 100. The condition may be termed a "trigger" event, which event is signaled to the routing proc- 
essor 202. The trigger event may be, for example, a router addition/failure or link connect/disconnect Control processor 
201 may also initiate the path selection process based on a predetermined schedule. The routing processor 202 pro- 
vides for an adaptive routing calculation, and also possibly new weighting calculations, to determine new routing provi- 
sioning entries and weights based on QoS-guarantee provisioning and network topology information stored within the 

w processing section 200. Provisioning entries define logical paths through the network links for different packet flows. 
[0046] FIG. 3 is a flow chart illustrating the steps of a router providing adaptive routing in accordance with the 
present invention based on an exemplary network supporting the OSPF protocol. At step 301, the adaptive routing is 
initiated by an internal event (such as by a predetermined schedule) or by an external event (such as by a trigger event). 
The detected event may initiate an optional step 302 in which the path-selection process determines a temporary set 

15 of provisioning entries according to the SPF routing method. At step 303, a multicommodity flow (MCF) routing method 
is employed for a path-selection process to determine new provisioning entries. The MCF routing method may assign 
some or all of the new provisioning entries based on the temporary set of provisioning entries. New provisioning entries 
are determined at step 303 based on optimization of the SPF routing calculations according to a predefined metric, 
such as optimized network revenue W. 

20 [0047] For example, at step 302 new routing based on QoS provisioning information and information provided from 
the provisioning entries of the SPF algorithm may provide an optimized set of provisioning entries for the new routing 
of classes and packets through network 100. New provisioning entries may group packet flows with a given identifier 
according to classes, with each class having defined QoS commitments. Bandwidth, or other QoS related commitments 
as metrics, of a given interface card may be allocated and then different classes assigned to the allocated bandwidth. 

25 [0048] At step 304, the current filter rules of each interface, or line termination, card are deleted for the packet filters 
that are changed or modified. At step 305, the currently defined packet flow identifiers (i.e., a value identifying the spe- 
cific packet flow, or aggregate of packet flows, that may be based on source and destination addresses) are deleted for 
each interface, or line termination, card for the packet filters that are changed or modified. At step 306, the current class 
definitions are deleted for each interface, or line termination, card for the packet filters that are changed or modified. The 

30 class definitions of each interface or line termination card specify which packet flow classes may be supported by avail- 
able bandwidth of the interface card. 

[0049] At step 307, bandwidth for each interface, or line termination, card is allocated based on, for example, the 
available bandwidth of each input fine interfaces 204 and the available bandwidth of each output line interface 207 (FIG. 
2). At step 308, new classes are assigned for the given bandwidth allocations determined in step 307. 

35 [0050] At step 309, the new classes generated in step 308 are installed in each corresponding interface, or line ter- 
mination, card. At step 310 defined packet flow identifiers for the classes are installed for each interface, or line termi- 
nation, card. At step 311 new filter rules are installed in, for example, the packet classifier 203. The filter rules are 
employed to route packet flows with corresponding identifiers from input ports to output ports of the router based on the 
actions associated with the filter rule. 

40 [0051] The new routing determined by the path-selection process is provided as provisioning entries of a routing 
table. The provisioning entries of the table are defined for each packet flow (which may be a packet flow or an aggregate 
of packet flows specified by source and destination address ranges) and a set of classes, each class defined by a set 
of QoS provisioning commitments. Each routing table for a packet flow has an associated class. An exemplary routing 
table for a packet flow based on QoS provisioning and network link topology is given in Table 3. 

45 
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Label 


Content 


src_rtr 


The identifier of the source router 


dst_rtr 


The identifier of the destination router 


srcjint 


The input logical interface of the source router 


dstjint 


The output logical interface of the destination router 


vpn 


The identifier for the client (or VPN) that the packet flow belongs to 


revenue 


Assigned revenue value 
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Table 3 (continued) 
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Label 


Content 


class 


Assigned class 


w2edelay 


Maximum end-to-end delay, in microseconds 


src_addr_low 


Lower bound of the source address range for a packet flow 


src_addr_hig 
h 


Upper bound of the source address range for a packet flow 


dst_addr_low 


Lower bound of the destination address range for a packet flow 


dst_addr_hig 
h 


Upper bound of the destination address range for a packet flow 
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SJfJL J!!??? h 3 ' th f src - addr - ,ow and src_addr_high may specify a source address range, 

In lLT 7 I f-S" dst - addr - hi 3 h ma V s*™«y * destination address range. These values may be 

zxiszzz i:r i flow - or a99resate of packet flows - consequen,,y - these four — ^ be 

ThifrL ^ 3PtiVe 7i T" accordance with ,he P resent "^ntion specifies a class to which the packet flow belongs. 
L of ' W,th 3 ^ ° f Q ° S commilmente »* are specified for the class. Table 4 provides an exempt 

MtoJSEZT"* C ° mmrtmen,S for 3 ,eaky bUcket re 9 ulator •» ■ Oass including a packet flow iden" 
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Label 


Type 


r 


rate in bytes per unit time 


b 


The sender bucket depth 


P 


Peak rate 


m 


minimum packet size (with overhead) 


M 


maximum packet size (with overhead) 



201 iich^nlT P t r ° VB,0n,n 9 entries ^sed on the trigger event are provided to the control processor 

Itl n H ^ ^ ? " 9 ° f source ^ estina «on pairs into filter rules. The filter rules are mapped as packet 

f daSSffier 2 ° 3 - " W0UW * aPPar6nt 10 °" e Skil,ed in the art - the P^ing section 
™™ exemplar y onl * and ma ny vanations on the processing section may be implemented 

Sftet filter 400 X irfn'^ "^rr implementation of the P actet ™« *>° °f Packet classifier 203 is shown in FIG. 4. 

£S L~ In? . T ^ S> ° f m,eS inC,Udin9 the ValUe 0f the ^ponding address fields of a packet. 
Packet fitter 400 includes a temporary storage register 402, an AND^perator 404, processing elements 406 and a 
memory 412 having interval date registers 408 storing one or more corresponding bftmaps 41 u as^atedlSh ^ 

fitter-rules associated wrth the intervals. a 
[0056] As shown in FIG. 4, a packet received by a router is stored in temporary storage register 402 which may be 
a buffer or regis exposition in a queue. Memory 412 stores interval data and corresponding oSmaps de ivS from fitter 

oSeST'l T S,n9 t e ! ementS 406 6aCh p- * nB 3 ^ — "* of the corresponding deSved inters for he 
packet fields. One .mplementation may employ one processing element 406 for each fitter-rule dimension (address 

aS XSEST ST"*' 3 State maChinS *"° l0Ca ' re9feterS - F0r this -Potion, the bina "siarctfor 
all stored intervals 408 by processing elements 406 is performed in parallel 

?T BS f ng !l 6 r nt 406 re,UmS 3 bttmap «»™»Pondlng to an interval including an address field of the 

are C ° mbined by the AND - ° perator 404 10 form a P°War. The resuS 

of this search process is a memory pointer to the "best-match" filter rule. The packet fitter then performs a further 
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access to memory 412 to retrieve actions associated with the identified filter rule or filter-rule set (not shown). The 
packet and corresponding retrieved actions are provided to the route allocation processor 206. In exemplary implemen- 
tations, a packet filter may test for between 512 and 800 filter rules. 

[0058] FIG. 5 shows a block diagram of an exemplary distributed processing and database (DPD) system 500 for 
5 alternate path routing in accordance with the present invention. DPD system 500 includes an OSPF processing module 
512 with corresponding link state database 510, QoS routing module 506, filter rule processing module 514, filter man- 
agement processing module 516, and packet classifier module 518. 

[0059] OSPF processing module 512 receives network topology information through the packet network interface 
in accordance with the OSPF protocol. Network topology information may typically include, for each router in the packet 

w network 100, a router identification address, identification for links connected to the router, other router addresses of 
other routers connected to the router, and information of other networks (e.g., N 1f N 2 , and N 3 of FIG. 1) that may be 
connected to the router. OSPF processing module 512 stores the network topology information in the link state data- 
base 510, and updates the information of the link state database 510 as new network topology information becomes 
available, such as when the network topology changes. The OSPF processing module further calculates routing infor- 

15 mation for the packet network in accordance with the SPF routing method known in the art. 

[0060] QoS routing module 506 receives tables having QoS provisioning information for source-destination pairs. 
The tables are received from the network management database 107 through network management interface 504. The 
QoS routing module 506 may also receive both the temporary provisioning entries of the OSPF routing method and the 
network topology information from the link state database 510. Given the QoS provisioning requirements for source- 

20 destination pairs, the QoS routing module 506 recalculates optimized routing of source-destination pairs through the 
network with the network topology information of the link state database 510. The QoS routing module 506 re-calcu- 
lates the optimized routing in accordance with the MCF routing method. 

[0061] Returning to FIG. 5, the DPD system 500 includes network management interface 504 that retrieves formats 
data of the network management database 109. Communication between network management interface 504 and net- 

25 work management database 109 may be mediated by a protocol such as Simple Network Management Protocol 
(SNMP). Data of the network management database 109 may be formatted into a record structure of provisioning 
tables with, for example, reduced entries and in a format that may be desirable for the QoS Routing module 506. The 
packet network interface 508 formats data received from control packets of the network into a format desirable for the 
OSPF processing module 512. 

30 [0062] The processing modules, database and interfaces of the DPD system 500 may be implemented in the 
processing section 200 of FIG. 2. For example, the network management interface 504, QoS routing module 506 and 
OSPF routing module 512 with link state database 510 may be implemented as software of the routing processor 202. 
The packet network interface 508, filter rule processing module 514 and filter management processing module 516 may 
be implemented as software of the control processor 201. The packet classifier module 518 may be implemented as 

35 hardware of the packet classifier 203. 

[0063] Several methods may be employed to provide shared use by the QoS routing module 506 and OSPF 
processing module 512 of the OSPF routing information and the topology information from the link state database 510. 
First, the QoS routing module 506 may receive and flatten the OSPF topology information from the link state database 
510 into a separately stored database for use by the QoS routing module 506. Second, the QoS routing module 506 

40 and OSPF processing module 512 may share a synchronized database access routine. Third, OSPF processing mod- 
ule 512 may be modified to recognize and employ network topology information from the link state database 510 stored 
in a manner consistent with the QoS routing module 506. Fourth, when the OSPF processing module 512 changes 
topology information from the link state database 510, the QoS routing module 506 may be notified and change its data 
structure in the same manner. 

45 [0064] Since the QoS routing module 506 calculates routing and path selection with QoS provisioning information, 
the QoS routing module 506 may require additional network topology information than that currently collected in accord- 
ance with the OSPF protocol. For this case, the control packets as currently employed by the OSPF protocol may be 
modified to allow for collection of additional features of network topology information. The payload of the OSPF protocol 
control packets includes a field defined for type of service (TOS) dependent, or addressed, metrics. This field includes 

so several TOS-addressed metrics. The TOS-addressed metrics identify TOS entries in a database or table, with each 
entry available for use with a TOS metric. Each TOS metric may be a form of control message available to the OSPF 
protocol. Not all addresses of the database or table of TOS entries of the protocol, however, are currently used for a 
TOS metric. 

[0065] For an embodiment of the present invention collecting additional network topology information, the control 
55 packets of the OSPF protocol may be modified to define unused TOS entries for adaptive routing. For example, the 
empty or undefined entries of the TOS-addressed metrics in OSPF packets may be defined to include information such 
as available link bandwidth (unassigned or assigned but not in use). FIG. 6 shows a line format of an OSPF packet 
showing the positions of packet type 601 and TOS bits 602, respectively. Consequently, a preferred embodiment of the 
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^LT^TS^ low- 

ZSZ^VX? one or more fie,ds of T0S metrics may * emp,oyed to ™ vey 

* fn°l 66] rH* FU k ^ °" 6 ° r , m ° re T ° S entneS may be em P loved t0 """V information related to routing protocol used 

3£T 7 y I 1 °°- ^ eXamP ' e ' thS TOS bte may te d6fined to identif y the typ- of 'oZ 

able from the router generating the packet, such as adaptive routing in accordance with the pLenTinvention Z slS 

SETS 0 ' ° f ^ meth ° d for *• P ath - sel ^°" P">cess. For example, paXte anSno Z 

network 1 00 may haye the defined TOS bits set to specify an unknown or default state. When a router e^Z Zt 

10 Z SP ,n nce the present invention rcceives such a packet 1,16 defined TOs bits are - 

fh!!IL . A r0Ute l rece i vin 9 such P acket with TOS bits set to the first state and employing adaptive routing (or knows 

ruchSSros [, "rf? 'rr ds the packet wrthout the ™ *■ a 

ZtZ t wrth TOS bits set to the first state that does not employ adaptive routing (or does not know the next logical 
is ^™trV? an9eS T ° S bitS 10 3 SeC0nd 81346 and fofward * *" a P-** A router receiving su* paS 

mTttSLl^ 2 St3te 'T VardS the P*** «■*"» for — "I* a path selected by a £5 

method for the path-selection process (i.e., the SPF routing method) 

theOSPF ?T emb ° di 4 me , nts of the P re ^nt invention may employ a signaling packet conforming to 

.>?„? SPF P rotocoL J ^ signaling packet allows a particular router to signal other routers in network 100 that the par- 

ton o!T^Z f ^ T" 9 aCCOrdanCe ^ the Present invention - Such -Y indicatfbo th a Z- 
swn or manufacturer, for example, and may be termed a QoS advertisement packet. The QoS advertisement packet 

in accordance with the present invention to p redict what 

^line 22.31. bV PartiCU ' ar rOUter aSSOCiated ™ th the QoS ^ertisement packet. FIG. 7 

m2 in^S on In TL afy adVert,Sement P acket includi "3 ^vertising router information 701. The advertising 

router information 701 may .nclude information related to the path-selection process and the routing protocol employed 
by the router originating the advertisement packet. P'oiocoi empioyea 

!2 l 2*pJ£ n * embodim e"ts of the present invention may include a method of ensuring that all routers of the network 
100 determine the same sets of provisioning entries by using similar network topology information. Such memod may 

infoZfior to^anS ' VT " add6d t0 the netW ° rk ' ° r * SOme other *«* t" a * toX 

so Sfo^S^^ 

[0070] The mismatch between the link state database 510 of each of the various routers of network 100 mav 
requ.re a distributed database commrt operation to synchronize the information of the database A ^SSSJSS 

^mT?l°« r^'V Si9nalinQ betWee " e ' ementS 0f a distributed databa * a to ensure that alfelemfnts aro 
modified according to the most recent network topology information. A distributed database commit operation mav 

mand). For the present .nvent.cn, such distributed database commit operation may require an exchang^o" paSete 
between routers of the network 100 to verify that each router has determined the same provisioning 
mrt ot^eratio^^^flnmpa ^ ^» the exemplary advertisement packet may be employed for the distributed database com- 
Z S^n t « ! broadcastln 9 Path-selection and other routing information determined by particular rout- 
ers, wh ch ,n turn reflects a state of the topology information in each link state database The advertisino router 
information 701 that may be employed to predict what routing calculation and path selection may be deterS byte 
particular router associated with the QoS advertisement packet. The QoS advertisement packet may 5 me 
provis.oning entry mformation for running and candidate sets of provisioning entries 705 and 70 ^ ^fprovision no 
entry informal may be included in a HASH table format. In addition, a woLg set of provisioning enttes 70TZ 

[0072] Returning to FIG. 5, once the previously described information is available, the QoS routing module 406 
rea!S^ 

real-time can tolerate delay) packet flows, which classification may be based on a delay threshold End-Lnd deSs 

a e c^^ delays (delay the ,ink be,ween ™*5 Reai -^ ™ SSSE 

SSffi JZl ™ m dth requ,rements assi 9 ned for ea ^ packet flow, and the packet flows are then further 
classified nto multiplexable or non-multiplexable flows, depending on whether the packet flow occupies more than a 
predetermined level of bandwidth of the link traversed. ««-up. e5 more man a 

55 En, Jrfnl* 3 ' aVa i' able C iS all0Ca,ed t0 the mu,ti P |exab ' e ^ nonmuttiplexable flows. Then, routing cal- 

^^1^ t "T de i emiined ** ^ multi P ,exable or "on-muttiplexable flows based on a corresponding 
55? orS^ n T mP ''J'! mUl,ipleXable flows ^ hava Election based on a murUcommod^y flow 
(MCF), or QoS-based, rout.ng method, such as TALISMAN, available from Lucent Technologies, Murray Hill. New Jer- 
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sey. The non-multiplexable flows may have path selection based on a lossless effective bandwidth algorithm, such as 
the SPF routing method. 

[0074] Adaptive routing in accordance with the present invention receives QoS information and network topology 
information mediated by the OSPF protocol to calculate the new routing tables. The calculation of new routing tables 
5 preferably completes within a time that enables existing packet flows to be re-routed before tear-down of established 
connections in accordance with a high-level protocol. To this end, some tables for re-routing may be pre-computed prior 
to use. 

[0075] QoS routing module 406 may employ a method for allocating resources and determining provisioning entries 
as shown in FIG. 8. First, at step 801, each packet flow is classified as a real-time flow or non-real-time flow based on 

10 the end-to-end delay of the packet flow compared to a predetermined delay threshold. Next, at step 802 per-hop, or 
nodal, delays d p are calculated for each packet flow (e.g., delay between routers). Per-hop delays d p are calculated by 
dividing the end-to-end delay for the packet flow by the maximum number of hops r?/^ for the packet flow. For real- 
time flows, iimax is one plus the minimum number of hops n M!N . For non-real-time flows, is a laxity factor plus the 
minimum number of hops n MiN . 

15 [0076] Once the per hop delays are calculated, at step 803 the effective bandwidth value ed 0 , of each packet flow 
is determined using the nodal delays dp and QoS information. For example, QoS information may be specified for a 
class the packet flow belongs to, and may include average token (packet) rate r, peak rate P, and the buffer size B token 
if a leaky bucket regulator is employed by the node. For example, if a node has a buffer size B^q and output link capac- 
ity Qrans' tne effective bandwidth value ed 0 of each packet flow may be calculated as in equation (1): 

20 




if Tbur £ T 0 ff 



ed Q = r, (1) 

30 

where ^~ buf~^ node ^ trans^ P n ® maximum delay time of the node s buffer), "^ 0 f^(B token r),and T on = B ioken 

(P-r))- 

[0077] At step 804 the packet flows are then classified as multiplexable (packet flows that may be mixed with other 
flow traffic on one or more links) or non-multiplexable (packet flows that are desirably assigned a dedicated link and 
dedicated capacity) based on comparison of the corresponding effective bandwidth ed 0 values with a multiplexing 
35 threshold. The multiplexing threshold may be calculated by, for example, an examination of the statistical properties of 
the network traffic flows at peak levels. 

[0078] At step 805 bandwidth is reserved from the total available capacity C for routing of packet flows exhibiting 
characteristics of high-volume traffic and non-multiplexable flows. High-volume traffic may be defined based on the 
maximum bandwidth (or peak rate P) of the packet flows. For these high-volume traffic and non-multiplexable packet 
40 flows, provisioning entries determined by the shortest-path routing of OSPF may be employed for routing of the packets 
through the node. 

[0079] At step 806 the residual bandwidth C MUX for multiplexable packet flows is determined. The residual band- 
width C MUX may be determined as the available capacity C minus the sum of the effective bandwidth values ed Q of the 
non-multiplexable packet flows. Alternatively, the residual bandwidth C MUX may be determined as the total available 
45 capacity C minus the capacity reserved for non-multiplexable packet flows. 

[0080] At step 807, an initial route is determined for each multiplexable packet flow based on requirement of the 
effective bandwidth value ed 0 , which requirement is determined by mapping the effective bandwidth value ed Q to 
allowed rates of the output links of the node. The initial routes may also be determined, for example, with the SPF rout- 
ing method. 

so [0081] At 808, routing for the multiplexable packet flows is calculated with a mufticommodity flow (MCF) routing 
method. The routing is provided as a table having a candidate set of provisioning entries. 

[0082] The method as shown in FIG. 8 employs the QoS provisioning information, the link state topology, and the 
packet flow classifications (real-time/non-real-time and multiplexable/non-multiplexable packet flows) to form provision- 
ing entries for routing of packets through the node (router). Such provisioning entries may define specific source and 
55 source port address ranges of packets to be routed to other specific destination and destination port address ranges. 
Further, the packet network 100 may employ a protocol, such as the Type of Service (TOS) bits and/or RSVP control 
messages, to allocate resource reservation over physical and logical links for the corresponding packet flows. 
[0083] The path-selection process of QoS routing module 506 provides an optimized routing of source-destination 
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mle P r0CesSI " 9 module 514 as a teble havi "9 ^ of candidate provisioning entries. The set of candi- 
date pensioning entries may be employed as information from which filter rules are derived. In accordance with the 

l^TT^f*™?^"" ^ ^ d6flned 50 35 10 roUte packets a -«*■ *»* of the router 

based on the newly determined path selection. Given the source and destination address ranges (and possibly the oro- 

"T""^ 0 !.' se,ected mer mle app,icable to a M >■** 

associated wrth the filter rule is applied to the packet. The action associated with a filter rule may be to route the 

^ P t et ^ haVin9 h6ader fiSld Va ' UeS inC,Uded in *• responding sourS and deafen 
address ranges from specific input ports to specific output ports of the router. 

of st!L a ^«!! r h ,8 r!! PriSe L a daSSifier Speciflcation and an action - ™* classifier specification defines ranges 
iTfc^ 6 ^" addreSSeS ' S ° UrCe P ° rt ran9eS ' and destination ^ ra "9es, and may be the packet 

?e a ^imenZJc V ' ^J"*.™ 188 3re instructed as rectangles in a four, five or six dimensional space 

,f a " ' nteger 9rcater th3n ° ne ""WxtoW to the number of unique fields used for packet 

routing). As would be apparent to one skilled in the art. higher dimensional spaces may be employed as the packet filter 

S fitter mte " ""k^ 16 ^ ™ y ^ determined from a priom V assianed <° 
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Label 


Type 


id 


The index of source-dest.nat.on pairs for the packet flow (note that when ranges are specified the 
packet flow may be an aggregate of packet flows) 


rtr 


The next-hop router that is employed for packet forwarding 


src_port 


The input port of the logical interface of the router 


dst_port 


The output port of the logical interface of the router 


bw 


The amount of bandwidth reserved for the packet flow 


b 


The size of the local buffer for the packet flow 



Z Th» , S TV fie,d th3t iS *" PaCk6t fl0W identifier ' and is associated with *e routing actions of the 
able. The entries of Table 5 may define actions such as cross-connection between logical port interfaces of the router 

ScT^Th n Z { TT- ^ W>mmitment for the Packet flow ' such as bandwidtnTnd b Se 

nTabte S 6 5 id6ntifieS SPeCifiC S ° UrCe and deStination address ran 9 es ' as sn °™ 
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Label — 


Type 


src_addr_low 


Lower bound of the source address range for a packet flow 


src_addr_high 


Upper bound of the source address range for a packet flow 


dst_addr_low 


Lower bound of the destination address range for a packet flow 


dst_addr_high 


Upper bound of the destination address range for a packet flow 



E I « e f ° f Tab ' e 6 maV fUrther indude the T0S deification and/or protocol identifier. 
entriS ThTSl'2 processin 9 modu ' e 514 ° f 5 creates the new filter rules based on these new provisioning 
module « 6 ZS. Pr0V,S ' 0n,n9 e t 6S and ""Wna fi« er rula * are then provided to the fitter management 
module 516, which also reta.ns cop.es of running filter rules corresponding to the currently used, or "running" set of pro- 
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visioning entries employed by the packet classifier module 518. A running set of provisioning entries, and correspond- 
ing running set of filter rules, are generated by the determined path selections prior to reconfiguration and routing re- 
calculation. 

[0089] The filter management module 516 compares the new and running sets of filter rules, and selectively deter- 

5 mines filter-rules to be deleted from the packet classifier module and filter rules that are to be modified. Once the 
selected filter-rules to be deleted or modified are determined, the filter management module 516 substitutes the 
selected packet filter rules into the packet classifier module 518. For the exemplary embodiment, the packet classifier 
module 518 may be employed by the packet classifier 203 (FIG. 2) to cause specific routing of packets within the routing 
fabric, such as by providing control signals provided to the route allocation processor 206. 

w [0090] As described previously, packet classifier module 518 receives filter rules from the filter management mod- 
ule 514, and applies the filter rules to the incoming packets. FIG. 9 illustrates the process of packet filtering on two 
packet fields, corresponding to a point location in 2-dimensions. A point location in 2-dimensions may be a location of 
a point within a particular filter-rule rectangle defined by source and destination address ranges. As would be apparent 
to one skilled in the art, the 2-dimensional point location maybe extended to higher dimensions, such as the 4-dimen- 

15 sional point location, with filter-rule rectangles further defined by source port address ranges and destination port 
ranges. As shown in FIG. 9, four overlapping filter rules operating on, for example, the source address field and the des- 
tination address field of a received packet are represented by overlapping 2-dimensional rectangles 901-904. The filter 
rule segments of the rectangles 901-904 are projected as intervals labeled X0-X8 and Y0-Y7 on the corresponding 
dimension axes X (for source address) and Y (for destination address). The intervals and bitmaps may be defined sim- 

20 ilarly to those described with respect to FIG. 4. 

[0091] In the example shown in FIG. 9, segments of the four filter rules in each dimension overlap the six intervals 
on the Y-axis and seven intervals on the X-axis, respectively. For a worst case, the projection results in a maximum of 
2n - 1 intervals on each axis for the dimension, where n is the number of filter rules. For the example, n is four filter 
rules, resulting in the worst case of seven intervals on the X-axis. Bitmap arrays 910 and 912 associated with each 

25 dimension are formed having bitmaps for each interval. A bit in the bitmap is set if, and only if, the rectangle correspond- 
ing to the bit position overlaps with the interval that the bitmap corresponds to. For example, interval Y1 has a corre- 
sponding bitmap 914. Since rectangle 901 overlaps interval Y1 , the bit at the first bit-position is set to logic "1". None of 
the rectangles 901-904 may overlap with, for example, only half an interval because of the method by which the inter- 
vals are defined. The bitmaps may be stored as one or more bitmaps having associated interval bitmap pointers. 

30 [0092] A packet arrives to the packet filter of the router. The packet may have several fields, but two field values, E-j 
and E 2 , one per dimension, form coordinates of a point EP for point-location in 2-dimensions. E-j may be a source 
address, and E 2 may be a destination address. Intervals are located on each dimension axis X and Y that include the 
corresponding field values of the point EP. In the example, these are intervals X5 and Y4 for the X-axis and Y-axis, 
respectively. The interval bitmaps 916 and 915 of intervals X5 and Y4, respectively, are employed to locate the highest 

35 priority rectangle that covers this point. The interval bitmaps 91 6 and 91 5 may be reconstructed from a retrieved bitmap 
and corresponding interval bitmap pointers. Rectangles 901-904 may be numbered and allocated bit-positions in the 
interval bitmap based on their priorities, with rectangle 901, for example, being the lowest priority and rectangle 904 
being the highest priority. After the combination of the interval bitmaps in step 904, the first bit set to logic "1" in the 
resulting filter-rule bitmap is the bit identifying the highest priority rectangle 904 amongst all those rectangles overlap- 

40 ping the point EP in the example shown in FIG. 9. 

[0093] For the preferred embodiments of the present invention, the QoS routing module 506 employs a MCF rout- 
ing method. However, the MCF routing method is derived from general routing optimization, and may be preferred if 
new, optimized routing is desirably determined in real time. The following describes the general routing optimization 
method, and then the MCF routing algorithm. The general routing optimization method includes methods based on both 

45 stochastic and deterministic routing optimizations. The general routing optimization method and the MCF routing algo- 
rithm are similar to those methods employed in network design tools. Such network design tools employ, for example, 
the method described in D. Mitra et al., "ATM Network Design and Optimization: A Multirate Loss Network Framework," 
IEEE/ACM Transactions on Networking 4 (August 1996) 531-543 (herein referred to as "Mitra et al."). The stochastic 
routing optimization and MCF routing methods of Mitra et al. may be employed with virtual private network design tools, 

so such as TALISMAN available from Lucent Technologies, of Murray Hill, N.J. The present invention is not so limited to 
the following method, and any routing method may be employed which calculates optimized routing paths through 
packet network 100 based on QoS provisioning information and network topology information. 
[0094] The general routing optimization method determines rates of traffic offered to each route between a source- 
destination pair that optimize performance (also known in the art as the optimum routing problem) of the network 100 

55 according to predefined optimization criteria that includes QoS provisioning information. The performance of the net- 
work may be characterized in various ways. For example, performance is desirably characterized by the value of the 
long-term network revenue W. Long-term network revenue W may be defined as the weighted measure of carried 
bandwidth on the network constrained such that each source-destination pair earns a minimum amount of revenue. 
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-w^ W iS ^ erm * n6d ^ the reV6nUe 6arned Per c " ri8d Connection P» unit time and the traffic rate (also 
referred to as Traffic intensity^. v 

[0095] The subscript 5 (s an integer greater than 0) represents each service type, and signifies that each service 
type has ite own particular set of values for these parameters. In the art of network design and management, the traffic 
5 charactenst.cs and measures of quality of service may be determined by the pertinent "application," and the term "serv- 
ice b used somewhat more narrowly. For the following, the term "service" is employed in a general sense in which it 
includes or is synonymous with the term "application". A class, or possibly a super-class, may be a service The sub- 
script r (r an integer greater than 0) represents each route in the set of permissible routes between a given source-des- 

«. m™*," S ? th3t eVe " Withi " 3 9iven service tyP 6 s - eacn route ha s «s own particular set of values 

10 0096] Given these definitions, a general routing optimization may be formulated as joint maximization of network 
revenue by considering all offered traffic to the network. FIG. 10 illustrates an exemplary flowchart for the general rout- 

2?2?T AtSt : p ;° 0 J" initial value8 are P rovided to the network topology and network traffic character- 
istics Jnrtial values include stream, or packet flow, identifiers, bandwidth, and port definitions for each node n of the set 
of nodes*, 1S»*Wandeac*lin*/ofmesetoflinl^ OoS 
is provisioning commrtments) for each service class s of the set of services S, 1 s s ss, the effective bandwidth ££5 
T, S ° n " nk * he demand for each Service class between eacn source-destination pair o, the 
ZTJT» fl0WS (S ' 0) ' 3nd ^P 3 ^ al,ocation °/ on the res P a *ive link /. Note that these initial 

values may be defined or determined as described with respect to FIG 8 

„ J° 097] h * ste P. 1 »02. an initial value for the network revenue W is provided. The value for W may be predetermined 
20 through network simulations, heuristic methods, or previous or current values measured prior to network reconfigure 
tion and/or re-routing calculation. a 
[0098] At step 1003, initial values are provided as current estimates for the traffic rates, The initial values for the traf- 
fic rates may be predetermined through network simulations, heuristic methods, or previous or current values measured 
prior to network reconfiguration and/or re-routing calculation. 

[0099] At step 1004, network performance of routing is determined from the current estimates for the traffic rates 
file inrtial value for the network revenue W, and the initial values for the parameters provided in step 1001 

£2!!L r ! . 1005 ' thS ValU6S ° f thS ° Urrent 6Stimates *• fraffic rates are incremented according to, e g the 
method of steepest ascent. At step 1006, a current value of W is calculated 

[0101] At step 1007, in an exemplary method of measuring the approach to a local optimum, convergence to the 
2 F5T«r5L ? r6VenUe W iS Perfbm,ed by COm P arin 9 tha difference between current and previous values 

to ™ J J£ ftT* ,S , *"? 3 Predefined threSh0ld ' the ito,aUvB procedure ™y be defined a * being converged 
to an estimate for the optimum value for IV. If W is less than the predetermined threshold, at step 1008 the current val- 

th^rt ™ ^hT 8 "* l Stim f S ^ the traffiC rat6S are provided as tne °P* imum °"tput values. The predefined 
threshold may be determined off-line through network simulation or other methods known in the art. Otherwise the pro- 
cedure is tented, by returning to step 1004, using the current estimates for the traffic rates as input values 

T^k 9 meth0d . iS d8riVed frDm 3 sim P |ifica « on to the general routing optimization with stochastic 
variables. Consequently, the general routing optimization with stochastic variables is first described. The general sto- 
chastic routing method derived with stochastic variables is as follows. The traffic intensity is defined as the mean arrival 
™^ a°" re , qU ? te ° ffered (L8 - t0 be estab,ished > t0 the given route multiplied by the mean holding period of 
I^IhTT' T > connection re 9 ues ts "ay follow a Poisson distribution. Related to the traffic intensity is the 
I ZTZ sr ° m r °" te ' f0f 3 S6rViCe C,aSS S - Tne loss P^bability is defined as that fraction of connections that 
ZZh r ? . ' ° r ^ ocked "- for a given link or route - Bl °<*ing occurs if any link in the route refuses to establish the 
connection due to insufficient resources to carry the connection on that link 

ITU/ 088 Pr0bab " itieS f mav exhibit ^Pendence on the offered traffic rates Psr Consequently, small changes in 
fraffic loading, n ,one portion of the network may have effecte over a larger portion of the network. L^sprobabilitieLay 
be determined from the relationship between link loss probabilities 6 S/ (probability of loss of service s on a link I) and 
route loss probabilities L sr (probabi.rty of loss of service s on route r). Therefore, an iterative method may be empfoyed 
to determine an estimate of the optimal set of offered traffic rates Psr by successive approximations. Successive approx- 
imation may employ an approach based on well-known techniques of optimization, such as the method of steepest 
ascent. According to this. nethod. knowledge of the sensitivity of network revenue W to each of the various parameters 

imSon, Tl T T&i t0 r0UteS (the ° ffered ^ rat6S te used ,0 formulate the next estimate (approx- 

E£2J nln n T T 3te ^^^tton). These sensitivities of network revenue W to each of the various 
parameters {dWldp sr ) are collectively referred to as the "network sensitivities " 

, arriving connection request of a given packet flow may be offered to any route in the set of permissible 
rf*?^ that Packet 1 tow These arriving connection request offers may be approximately distributed over the routes 
InnZf Lf 9 i° respec,ive ' Predetermined probability values. That is, for each route, the probability that a 

™^ C ° n, ! eCt,0n Wi " b6 0ffered 10 that route has 3 Predetermined value. The optimized values* the 

parameters Psr are employed to determine these probability values. 
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[0105] General routing optimization with stochastic variables is now described referring to FIG. 10. At step 1001, 
the initial values are provided to define the network topology and network traffic characteristics, and, at step 1002, an 
initial value for the network revenue W is provided. At step 1003, an initial set of values are provided as current esti- 
mates for the traffic rates, p sr 

5 [0106] At step 1004, route loss probabilities L sr and network sensitivities dW/dp sr are determined from the current 
estimates for the traffic rates p^, the initial value for the network revenue W t and the initial values for the parameters 
provided instep 1001 

[0107] At step 1005, the values of the current estimates for the p^ are incremented according to, e.g., the method 
of steepest ascent. At step 1006, a current value of W is calculated as in equation (2). 

10 



^=1 Z e„M\-L„) (2) 

15 



[0108] At step 1007, convergence to the optimum value of network revenue W is determined. If the difference is 
20 less than a predefined threshold, the iterative procedure may be defined as being converged to an estimate for the opti- 
mum value for W. If W is less than the predetermined threshold, at step 1008 the current values of W and the current 
estimates of p^ are provided as the optimum values of W and p^. Otherwise, the procedure is iterated, by returning to 
step 1004, using the current estimates of p^ as input values. 

[0109] At step 1004 of FIG. 10, the performance of the determined routing and network sensitivities are determined 
25 by calculating the loss probabilities and the network sensitivities dWIdp^. The network sensitivities may preferably 

be defined as the respective derivatives of network revenue with respect to the offered load of service types on route r. 

The link loss probability B s! of each service type s on each link / may be generated at step 1004 of FIG. 10. Procedures 

known in the art may be employed to derive, from the link loss probability B st and other information generated in the 

iterative procedure, a measure of the desired bandwidth utilization for each link of each service route. The desired 
30 bandwidth utilization may then be employed to determine nominal allocations of bandwidth for each packet flow, or 

packet flow, to each link of each service route. Such procedures are described in, for example, Elwalid et al. and M'rtra 

etal. 

[01 10] The step 1004 of FIG. 10 for general routing optimization with stochastic variables is shown in FIG. 1 1 . The 
step 1004 of FIG. 11 is shown as a method of five steps, shown respectively as steps 1101, 1102, 1103, 1104, and 

35 11 05. At step 1 101 , a self-consistent solution of two systems of coupled equations (for each service class s and link I) 
is solved for the predefined effective bandwidths ed^, the predefined link capacities C t , and the current estimates of the 
offered traffic loads p^. The step 1101 may include a determination of the link loss probability of calls of service s 
on link /. Since each route r carrying traffic of service type s and using link / adds the load v s/T on link / (which added 
load may follow a Poisson distribution), the added load may be at a rate that is reduced by independent traffic thinning 

40 (i.e., dropping of connections) by all other links in the route. By summing the added loads v s/T over all routes r which 
use link /, the total thinned, or reduced, load of service s offered to link / is v s/ . The reduced load with link independence 
is given in equation (3): 



Vrf-X X Csr IT < 1 - e sm> (3) 

a reR{s,a)ier mer-{l) 



[01 11] In addition, the B s/ may be determined from equation (1) and expressed as a mapping <|> s / of the set of total 
50 thinned loads per link 

55 for all services s (1< s < S). 

[01 1 2] The step 1101 may employ an interative method to solve these coupled equations, referred to as fixed-point 
equations (FPE). For the set of all routes R(s,a), one system of equations defines the set of link loss probabilities B sl 
and is given in equation (4), and the other system defines the set of offered loads v s/ and is given in equation (5). 
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&/=Mv,)(l< s <S) (4 ) 
v = «F(B) (5) 

where 

(for all links / (1 S / s L)), and 

KlinksMlV/^ t) S ~ S) an<J " nkS ' (1 L)) ' * fe maPPin9 S6t { * s ' )s ' for a " SenkBa s l^s*S) and 
2f [01 13] At step 1102, the route loss probabilities L sr are determined from the relationship as given in equation (6): 

^r^-n^s/) (6) 
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10114] Step 1103 then determines the derivative of the link loss probability e s , with respect to each of the total 
thinned loads v s , as given in equation (7): 



= ( l- B„)[Utd,,vuCrd t ) - Z,,(ed,,v,,C/) (7 ) 



t^nnL' ? a ,t m 7 Variable ind6X ' 6d ' iS the SSt ° f 6ffeCtive bandwidth s {ed s/ } for all services s on link /, v, is the set of 
thinned {v s/ } for all services son link/, and C, is the capacity of link/. 

f!! 1 !L K At St t P V" 1 ° f P arameters c *< is calculated for each service class a and each link /. These parame- 
ter^ which may be defined as implied costs, reflect the effective loss of revenue occasioned when the carrying of con- 

■SZ ZiZZJT^tT" V" 1 9lVen ' ink ' redUCes the remainin 9 ca P ad * and thus leada to an incremental 
uZ Zl ° f future offered cal,s " The P re °efined network revenue earned per carried connection per 

unit time, and the route loss probability Lsr , which is the equilibrium probability that after admission to the network a 
connexion of service class S will be dropped by route r (a route will drop a call if any of its links refuses the call) are 

in ^uSo„ S Zl Va , h 6P 1104 - At St6P 1104> the P3rameterS C * are determined from ,he Unship as given 
in equation (8) (where /is a dummy variable index overs): 



V »,/./ «r.. r-SI(.,.ff,/c, 0%, ^ k*r-M 



of i 8) repre f I"' 5 3 SVStem ° f(S X L) linear equations ma V be s 01 ^ ** *e set of parameters Csl . The com- 
em^^ 

SUhin rJ«, h ICUla !i 0 ,l ° f ?! tW ° rk sensitivities W/3 Psr is performed at step 1105. The network sensitivities dWld Psr 
are then read.ly derived from the route loss probabilities Lsr , the earnings parameters esr , and the implied costs c s/ as 
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shown in equation (9): 



(9) 



[0117] In many cases of practical importance, the link capacities may become so great that it is not economically 
or numerically feasible to solve the FPE as given by equation (4) and equation (5) to find the implied costs by exact tech- 
niques. If the calculation employs exact techniques, the calculation of the link loss probabilities may have complexity 

w O(C), where C characterizes the link capacities. 

[0118] in such cases, as described in Mitra et al., an asymptotic approximation referred to as the Uniform Asymp- 
totic Approximation (UAA) method may be used to reduce the complexity of the calculations. In addition, various values 
for groups of small links may be determined by exact techniques while, concurrently, various values for groups of larger 
links may be determined by the UAA method. Such an approach may be termed a "hybrid" approach. 

15 [01 19] When the UAA method is employed, solving the fixed-point equations may require less computational com- 
plexity, and hence may require less time to perform the computations. The complexity of calculating the link loss prob- 
abilities may be reduced to 0(1 ) (i.e., it remains bounded as C increases without bound), and the complexity of finding 
the implied costs may be reduced to 0(L 3 ). Thus, the complexity of finding the implied costs is made independent of 
the number of service classes. This is especially significant for large multirate networks that serve customers with 

20 diverse needs and thus carry traffic in many different service classes. 

[0120] Although the UAA method may be preferred for the stochastic optimization as addressed in Mitra et at., dif- 
ferent asymptotic approximation techniques may be employed when solving the stochastic optimization in the context 
of the present invention, such as the Refined Uniform Asymptotic Approximation (RUAA) method as described in J.A. 
Morrison, K.G. Ramakrishnan, and D. Mitra, "Refined Asymptotic Approximations to Loss Probabilities and Their Sen- 

25 srtivities in Shared Unbuffered Resources," SIAM J. APPL. MATH. 59 Number 2, November 1 998, 494-51 3. The RUAA 
method may be preferred as more accurate for calculating implied costs and sensitivities to link loss probabilities. 
[0121] However, since the present invention relates to adaptive routing in which new network routing is determined 
in real time, the stochastic optimization of the UAA and RUAA methods may not necessarily provide adequate perform- 
ance. Consequently, the preferred embodiments of the present invention employ the MCF routing method as derived 

30 from a simplification to the general routing optimization with stochastic variables. The simplification removes the char- 
acteristics of randomness and stochastic variables from the stochastic system. The connection request arrival and con- 
nection holding times are considered as deterministic variables (deterministic traffic rates defined as x sr )» wn ' ,e tne 
packet flows are considered as static or infinite duration fluids or flows. With this simplification, the optimum routing 
problem is considered as a linear programming problem (a large system of linear functions of unknowns and linear 

35 objective function). 

[0122] General routing optimization with the MCF routing method is now described referring to FIG. 10. At step 

1001, the initial values are provided to define the network topology and network traffic characteristics, and, at step 

1002, an initial value is provided for the network revenue W, approximated by maximized deterministic revenue W F 
(defined below). At step 1003, an initial set of values are provided as current estimates for the traffic rates, x S r Tne '™- 

40 tial set of values for the traffic rates x sr may be predetermined through network simulations, heuristic methods, or pre- 
vious or current values measured prior to network reconfiguration and/or re-routing calculation. 
[01 23] At step 1 004, the performance of the network routing and network sensitivities are determined from the cur- 
rent estimates for the traffic rates % sn the initial value for the network revenue W F , and the initial values for the param- 
eters provided in step 1001. 

45 [0124] At step 1005, the values of the current estimates for the x S r are incremented according to, e.g., full pricing. 
At step 1006, a current value of W F is calculated. 

[0125] At step 1007, in an exemplary method of measuring the approach to a local optimum, convergence to the 
optimum value of network revenue W F is performed by comparing the difference between current and previous values 
of W F . If the difference is less than a predefined threshold, the iterative procedure may be defined as being converged 
so to an estimate for the optimum value for W F If W is less than the predetermined threshold, at step 1008 the current val- 
ues of W F and the current estimates of Xsr are provided as the optimum values of W F and Xsr Tne predefined threshold 
may be determined off-line through network simulation or other methods known in the art. Otherwise, the procedure is 
iterated, by returning to step 1004, using the current estimates of Xsr as input values. 

[0126] At step 1 004 of FIG. 1 0, the performance of routing and network sensitivities with respect to maximized rev- 
55 enue W F are determined by solving the linear programming problem, as given by equation (10) with the constraints of 
equation (1 1) and equation (12): 
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Max0>=]T £ e sr x sr (i 0 ) 



X Xsr ^ P«r f° r all 5 and cr (U) 



^ ^,x ir £ G for all / (ig 
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m,Zt£ ( ?' T ! ° ; ! nd eqUat, ° n (12) ' th6 ValUe ^ 5 0 for a " s and and is the a ^ge (median or 
mean) traffic demand As would be apparent to one skilled in the art, W F , where W F is network revenue according 
to the solution of the linear programming problem, and W is the network revenue as determined from equation (2) 
20 [0127] Linear programming techniques are well known in the art, and simplex-based or interior point-based tech- 
niques, for example, may be employed for a solution for the present invention. For example, simplex-based techniques 
are descnbed ,n David G. Luenberger, Linear and Nonlinear Programming; Addison-Wesley, 1984. Interior point-based 
techniques are descnbed in N.K. Karmarkar and KG. Ramakrishnan, Computational results of an interior point algo- 
rithm for large scale linear programming, Mathematical Programming 52 (1991) 555-586 

!! 128 i, ♦ !f nsitivities ma y P referab| y te defined as the respective derivatives of network revenue with 

respect to the offered traffic rates Xsr of service type s on route r. Solving the linear programming problem for network 
performance wrth respect to maximized revenue W also provides the values for the network sensitivities. Procedures 
known ,n the art may be employed to derive from the offered traffic rates and network sensitivities a measure of the 
desired bandwidth utilization for each link of each service route. The desired bandwidth utilization may then be 
employed to determine nom.nal allocations of bandwidth for each packet flow, or packet flow, to each link of each serv- 
ics routs. 

JP129] While the exemplary embodiments of the present invention have been described with respect to methods 

LTf °" ' S f S ° ' imited - * W0U ' d te 3pparent 10 ° ne skil,ed in art - various ft""*"* may also be 
implemented in the circuits or a combination of circuits and in digital domain as processing steps in a software program 
35 of, for example, a digital signal processor, micro-controller or general purpose computer 

ES-JL!* ? furthe ^ unde f 000 that ™i°"s changes in the details, materials, and arrangements of the parts 
which have been descnbed and illustrated in order to explain the nature of this invention may be made by those skilled 
in the art without departing from the scope of the invention as expressed in the following claims. 

40 Claims 

1. A method of routing packets through a router of a packet network comprising the steps of: 

a) collecting 1) network topology information and 2) quality of service (QoS) provisioning information for each 
packet flow through one or more routers of the packet network; 



45 



b) determining a path for each packet flow using a general routing optimization method based on the QoS pro- 
visioning and network topology information; 

so c) generating a set of one or more filter rules for the router based on the one or more paths for the one or more 

packet flows passing through the router, each filter rule defining a physical path for one or more packet flows 
through the router; and 

d) applying a selected filter rule to each packet of a packet flow to cause each packet to traverse the corre- 
55 sponding path through the router. 

2. The method as recited in claim 1 , wherein the general routing optimization method is a multicommodity flow (MCF) 
method, and step a) includes the steps of: 
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a1) receiving, in accordance with a network protocol, a potion of the network topology information from each 

router of the packet network into a database of the router; and 

a2) receiving the QoS provisioning information from a network management database. 

5 3. The method as recited in claim 2, wherein, for step a1), the network protocol is at least one of a link-state protocol 
and a topology-covering protocol. 

4. The method as recited in claim 2, wherein, for step a1 ), the network protocol is the open shortest path first (OSPF) 
protocol. 

10 

5. The method as recited in claim 1, wherein step b) includes the steps of: 

b1) computing the effective bandwidth of each packet flow; 

15 b2) classifying each packet flow as a multiplexable flow or a non-multiplexable flow based on a comparison of 

the corresponding effective bandwidth with a threshold value; 

b3) reserving a first portion of an available capacity of the router for the non-multiplexable flows and a second 
portion of the available capacity for the multiplexable flows; 

20 

b4) determining a set of candidate routes, each candidate route corresponding to a packet flow, and at least 
one candidate route allocated to the second portion of the available capacity ; and 

b5) calculating the paths based on the set of candidate routes. 

The method as recited in claim 5, wherein step b) further includes the steps of classifying each packet flow as a 
real-time flow or a non-real-time flow based on a delay threshold, and calculating a nodal delay for each packet flow 
based on the network topology and corresponding classification as a real-time flow or a non-real-time flow. 

30 7. The method as recited in claim 6, wherein step b) further includes the steps of determining one or more candidate 
routes for the non-multiplexable flows in accordance with a network routing method; and calculating a correspond- 
ing path for each candidate routes of the non-multiplexable flows. 

8. The method as recited in claim 7, wherein, for step b), the network routing method is a shortest path first (SPF) 
35 method. 

9. The method as recited in claim 6, wherein step b4) determines the set of candidate routes by the steps of: 

i) receiving a present value for network revenue and estimates for offered rates of traffic for the one or more 
40 packet flows of each class, the class identifying a set of QoS commitments of the QoS provisioning information 

associated with one or more packet flows assigned to the class; 

it)determining route loss probabilities and network sensitivities of each class based on the nodal delay and 
effective bandwidth of each corresponding packet flow, the network topology information, the present value for 
45 network revenue, and the estimates for offered rates of traffic; 

iii) adjusting the estimates for the offered rates of traffic; 

iv) forming a new value for network revenue; 

50 

v) comparing the new and present values for network revenue to determine whether the new value converges; 
and 

vi) if the new value converges, determining the set of candidate routes based on the new value of network rev- 
55 enue and the estimates for the offered rates of traffic; otherwise, setting the new value of network revenue as 

the present value and then repeating steps ii)-v). 

10. The method as recited in claim 1, wherein step c) includes the steps of: 



25 

6. 
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d ) deleting a set of running filter rules from at least one interface card of the router; 

c2) deleting each packet flow identifier and each class currently installed in each interface card, wherein the 
packet flow identifier identifies one or more packet 

1 1. An apparatus for routing packets in a packet network comprising means adapted to carry out each step of a method 
as claimed in any of the preceding claims. 

12. A router of a packet network routing packets in accordance with an internet protocol, the router comprising: 

a routing processor comprising: 

an Open Shortest Path First (OSPF) processing module adapted to receive, in accordance with an OSPF 
routing protocol, network topology information from each router of the packet network into a link state data- 
base of the router, 

a link state database adapted to receive QoS provisioning information from a network management data- 
base into the link state database, and 

a QoS routing module adapted to determine a path for each packet flow using a multicommodity flow rout- 
ing method based on the QoS provisioning and network topology information; 

a control processor adapted to generate a set of one or more filter rules for the router based on the one or more 
paths for the one or more packet flows passing through the router, each filter rule defining a physical path for 
one or more packet flows through the router; and 

a packet classifier adapted to apply a selected filter rule to each packet of a packet flow to cause each packet 
to traverse the corresponding path through the router. 
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FIG. 3 
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